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ABSTRACT 



This workbook is designed to introduce service learning 
practitioners to the evaluation process and to help practitioners generate a 
local support network. The design portion guides the practitioner through a 
self -directed study for their own program and is formatted for use with or 
without a formal training session. The workbook is divided into two parts of 
three sections each. What follows is a list of 22 resources that assists 
users in doing research, particularly qualitative studies. Part 1 guides 
production of a needs analysis which defines information to be gathered, the 
audience who will use the information, and expected outcomes of using the 
information. A formal study plan is then written using the needs analysis in 
part 2. Two appendices provide additional tools for prioritizing study 
questions and determining the method of collecting and organizing data. Part 
2 also provides workbook examples and a sample plan to stimulate thinking. 
The final section of the workbook reports actual workshop results. (EH) 
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DOING SELF-DIRECTED STUDY FOR SERVICE-LEARNING 



The purpose of this project is to improve the practice of service-learning through better 
evaluation. One important way to get better knowledge is to engage those closest to the 
field, practitioners, in the evaluation process. To achieve this goal, administrators, 
teachers, and community members involved in service-learning can learn to plan and 
implement self-directed studies. To accomplish this end, the following program is 
designed to introduce practitioners to the evaluation process. To support their effort, 
additional assistance is needed to provide them with feedback and technical help to 
implement their evaluations, and to generate a network to sustain the evaluation 
process. Our goal is to work with resources available in the state to develop this 
support network. 

The materials contained in this workbook are intended to help practitioners think about 
the evaluation process. They are not intended to stand alone as a single source of 
information; rather they are designed to be used with other people, books, and 
information sources to begin the process of developing "reflective practitioners." 
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SELF-DIRECTED EVALUATION PLAN 



Preparing a self-directed evaluation plan is a process. It begins by determining 
questions which you want answered about yourself and your program. It develops 
as you determine how to answer the questions, what kind of information you need, 
who will use that information, how to gather and process the information, and how 
to analyze the information to answer your initial questions. It ends when you draw 
conclusions about what was learned from the effort and then use that information 
to improve some aspect of the program. That use stimulates more questions and 
the process begins anew as you develop additional questions about the changes that 
have been made. In other words, the process never ends; it simply recycles itself. 

The following pages are intended to help you begin the process of designing a self- 
directed study. By completing each exercise you will end up with an initial plan for 
your investigation. Be reminded that this is simply your first attempt; it is most 
likely you will revise it several times before you settle on a final draft. So do not be 
discouraged if it is not perfect by the end of the day. Good plans don't just happen, 
the evolve slowly. Good luck and good planning! 
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I. THF. QUESTIONS 



Your self-directed study begins with your own questions. What is it that you want 
to learn about your program? Reflect on your program, and write those questions 
which you feel are the most important to ask. For starters, write one question for 
each area listed. 

a. About participants in the program 

b. About people affected by participants 

c. About products produced by participants 

d. About institutions/organizations involved with your program 

Questions Asked - Suggested 



a. 



b. 



c. 

d. 



Next, write four questions of your own choosing covering any topic you desire. The 
important point is that you identify questions which you are seriously interested in 
pursuing. 

Questions Asked - My Own 
1 . 

2 . 

3. 

4. 




Cs 



2A 



PRIORITY LIST 



Now that you have identified questions important to you, place them in order of 
priority. Examine your questions and pick the top four in order of interest and 
importance to you, that you would most like to answer (see Appendix A if you 
would like to use your program goals to help prioritize your list). 

1 . 



2 . 



3. 



4. 



QUESTIONS - IN ORDER OF PRIORITY (REVISED) 

Now is the time to reevaluate your list, focusing in on just two questions which will 
serve as the major emphasis of your self-directed study. You can pick questions 
which measure your goals, or you can choose other options available. No matter 
what, identify the two most important questions which can become the focus of 
your self-directed study. 

1 . 



2 . 
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II. INFORMATION AND THE AUDIENCE 



Now that you have some questions to begin the study, consider what kind of 
information answers the questions and who might use the information to further 
improve the program or make changes which justify doing the study. Once you are 
clear about what you should know, it is easier to discuss who can use the 
information and what purpose the information can serve. 

Information Needed To Answer My Questions 



INFORMATION USERS - HOW THEY CAN USE THE INFORMATION 



Organizing the information allows you to project how it can be used by others. As 
you imagine how the information will look once it is organized, list those 
people/ organizations who can use the information for some important purpose. 
Do not forget to list yourself as an information user! 



Primary Information User 



Flow They Can Use the Information 



INFORMATION TO BE USF.D AND HOW IT IS TO BE REPORTED 



After you have decided what information can be used and who can use it, 
determine how you intend to report your information. Be sure that the 
information gathered and reported has value, first and foremost, to yourself. 

Information to be Used How to be Reported 

1 . 



2 . 



3. 



4. 



INFORMATION TO BE USED AND EXPECTED OUTCOMES FROM ITS USE 

As you think about the information gathered and those primary information users, 
ones who are going to do something with the information you collect, think about 
expected outcomes for use of the data. What improvements or changes do you 
anticipate as people use the information from your study? 

Information Used Expected Outcomes 

1 . 

2 . 

3. 

4. 
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ANALYSIS OF INFORMATION GATHERED 



In deciding how to report the information collected, you are actually determining 
how to evaluate your data so you can make sense out of it. This is critical to your 
study because it indicates how the information you gather should be interpreted and 
read. It also tells others exactly what the scope of your study is and the terms by 
which you expect to be judged. This is a most important step. 

In your method of analysis section, explain what you intend to do with the data so it 
can be conveyed to others in a clear and understandable manner. If your data is in 
the form of numbers or counts, use a method which reflects the quantitative nature 
of your information. If, on the other hand, your data comes from observation, 
interviews, etc., use a method which reflects qualitative dimensions, such as case 
studies, reports of observations, discussion of patterns or themes, etc. It is possible to 
use both approaches; in fact, it is even desirable to do so! 

Information Available Method of Analysis 

1 . 



2 . 



3 . 



4 . 
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III. DEVELOPING THE SELF-DIRECTED STUDY PLAN 



Now that you have most of the pieces of your study outlined, it is possible to put 
everything together to develop your self-directed study. Review your purpose and 
goals, your questions, who and what is being studied, how you will gather 
information, who will use the information, how you will analyze the information, 
and finally how you will report this new information to yourself and others 
concerned about improving their understanding of service-learning. It is easiest to 
lay out your plan in a linear fashion using the matrix presented below. As you 
complete the various items, check for consistency. Review the following issues as 
you evaluate your plan: 

1. Do the questions address the key issues you want to study (and deal with 
important goals of your program)? 



2. Is the information you are gathering useful in answering the questions? 



3. Will the information allow you to say something important about your 
questions? 



4. Will the method you use to gather information produce the quality and 
quantity of evidence to draw some kind of conclusions or, at least, 
make some statements as to what is happening in your program? 



5. Will information users (including yourself) be able to understand your data 
and reach the same conclusions you do about the evidence? 



6. Will the results of the study make a difference in how you conduct yourself 
or your program in the future? 



7. Is the plan something that can be realistically done? 
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This last question is perhaps the most important, because if the plan is too broad, too 
extensive, or too difficult, chances are you will never complete the study. It is easy 
to construct good plans which bear little resemblance to reality and then just put 
them on the shelf and never complete them. 

This is exactly what you are not supposed to do in a self-directed study. The plan 
must be something that can be done with little extra effort or outside support. It 
should be woven into the fabric of your existing program, with data collection, 
evaluation, interviewing, etc. a normal part of the existing program. To require too 
much extra time or effort will probably doom your study. 

Given this limitation, it is strongly advised that you do what is normally done in 
the context of the program. This means if you already evaluate participants along 
certain measures or criteria, you may wish to consider using that evaluation process 
as an integral part of your self-directed study. If you already talk to participants on a 
regular basis to understand what they are doing and why, then continue on with 
existing methods and interject some of your own questions or concerns into the 
normally occurring dialogue. This way you will be covering the information 
needed, but in ways that use your existing program activities to get at your research 
agenda. 

With this advice, please plan your self-directed study on the outline provided. Once 
it is written, you can evaluate it to determine whether it meets your goals and is 
really something that can be accomplished this year. 
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Date: 



SELF-DIRECTED EVALUATION PLAN 

Research Questions 



Information Needed 



Information Users 



Method of Collecting Information / Data 



Data Analysis 



APPENDIX A 



USING PROGRAM GOALS TO PRIORITIZE QUESTIONS 

Once you have your priorities established, do some serious reflection on which 
question or which questions would be most appropriate to investigate this year. A 
helpful way to decide which one(s) are perhaps most important is to examine the 
goals of your program. Ask yourself, "What am I trying to accomplish and what 
outcomes do I hope will occur as a result of my program?" Try to identify four goals 
of your program. 

Program Goals 

1 . 



2 . 



3 . 



4 . 
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CONNECTIONS BETWEEN QUESTIONS AND COALS 



One way of determining your priorities is to view your questions in terms of your 
program goals. Are there questions you want to investigate which address your 
goals? Look at the list of questions and program goals and identify the connections 
between the two. 

Write down the questions you have identified and the corresponding goals which 
they address. 



Questions 



Goals 



APPENDIX B 



METHOD OF COLLECTING INFORMATION 



After you have determined the information you need, figure out how you are going 
to collect it easily and completely. Review the needed information and describe the 
steps you will take to actually gather the data. Consider the steps involved in 
collecting the information and in organizing it for the information users identified 
earlier. See the sample sheet if you have questions. 



HOW I WILL ORGANIZE THE INFORMATION I COLLECT 



Once you have decided how to collect the information, consider how you will 
organize it to make sense for yourself and for others. Making sense means analyzing 
the information and packaging it in such a way that others can understand your 
process of thinking. Report here how you think you will organize/package the 
information so others can understand. 



1 . 



2 . 



3 . 



4 . 
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SELF-DIRECTED EVALUATION PLAN 



PART n - SUGGESTIONS AND SAMPLES 



This guidebook is a companion to the "Self-Directed Evaluation Plan" used 
for the workshop. Its purpose is to provide samples and suggestions for 
various assignments you will be working on as you do your self-directed plan. 
Please refer to these examples for guidance. They are designed solely to 
stimulate your own thinking on the subject and not to restrict your 
imagination. Remember — you need to prepare a plan that fits your interest, 
your program, and your situation! 





I. THE QUESTIONS 



This section is designed to focus your attention on what you want to learn 
from the self-directed study. The emphasis is on questions you want to ask 
about yourself, your program, and the interactions between all the people and 
institutions involved in your program. Here are some sample questions 
which address the four areas outlined on the worksheet. 



Questions Asked - Suggested 



a. How can I more effectively monitor student learning in the community so I 
can be sure students are learning academic subjects? 



b. What do people at the nursing home and the elementary school say our 
students really contribute through our service-learning program? 



c How can I show that students learn more math and science through 
participation in service-learning project? 



d. What real benefit is there for the community agencies which participate in 
our service-learning program? 



Questions Asked - My Own 



1. How c*n I measure what students learn through their service-learning 
activities? 



2. How can I show that students are more interested in school as a result of 
their service-learning experiences? 



3. What concerns or questions do parents have about involving their children in 
service-learning? 



4* How does service-learning motivate my students to do better in school? 



PRIORITY LIST 



On this section you need simply to review your questions, such as the ones 
stated above, and pick four (sounds like the race track)! Here are the four I ve 
chosen as most important to me. 



a. How can I more effectively monitor student learning in the community so I 
can be sure students are learning academic subjects? 



c How can I show that students learn more math and science through 
participation in service-learning project? 



2. How can I show that students are more interested in school as a result of 
their service-learning experiences? 



3. What concerns or questions do parents have about involving their children in 
service-learning? 



QUESTIONS - IN ORDER OP PRIORITY (REVISED) 

Now is the time to select the two questions which you most would like to 
investigate this year. You can develop plans around one or both. Make sure 
that the question or questions you pick are ones of great interest to you. The 
investigation you are about to undertake is for you, primarily, so there must 
be a strong desire on your part to spend some extra time discovering what it is 
you want to know. Here is my priority list. 



I. How can I more effectively monitor student learning activities in the 
community so I can be sure students are learning academic subjects? 



II. How can I show that students learn more math and science through 
participation in service-learning projects? 



These two questions are the ones I would want to investigate today. The two 
you select will serve as the basis for the planning exercises which will follow. 



II. INFORMATION AND THE AUDIENCE 



Review the two questions listed above and think about the kind of 
information necessary to answer the questions. 



Information Needed To Answer Mv Questions 



I. For this question I would need information about what students learn at their 
community sites. I would especially need information about applications of 
academic subjects. 



II. For this question I would need to get information about applications of math 
and science through the community service activities. I would also need to 
get information about science and math learning from students who didn’t 
do service-learning so I could possibly compare the two groups. 



INFORMATION USERS -- HOW THEY CAN USE THE INFORMATION 



In this section, think of the people/organizations who can use this 
information for some purpose. Doing evaluation has little value unless 
someone can use the information for a positive purpose. Identify those who 



can use it and how it can be used! 




Primary Information User 


How Thev Can Use the Information 


1. Myself 
(I) 


I can use it to award academic 
credit to students; to show my 
principal that service-learning 
does have merit in the academic 
program; to improve my 
understanding of the relationship 
between classroom learning and 
experiential learning. 


2. Principal 
(I) 


To show the school board that our 
service-learning program should 
be expanded as part of the 
academic offerings at the school; to 
help other teachers learn to do 
service learning activities with 
students. 


3. Myself 
(II) 


To increase the amount of math and 
science credit awarded through off 
campus programs; to demonstrate to 
other math/science teachers that 
students can learn effectively in the 
community. 


4. Curriculum committee 
(II) 


To generate new offerings in the 
district which include math and 
science credit from service 
experiences; to train other teachers 
how to develop service-learning 
programs in math and science. 



INFORMATION TO BE USED AND HOW IT IS TO BE REPORTED 



This section requires you to think about how your information will be 
reported to the information users. It will help define the method and 
approach to data assessment and reporting. 

Information to be Used How to be Reported 



L Reports from students 
and community sponsors 



2. Reports from college 
students 



3. Test scores from 



4. Portfolios of student 
work in science/math 



Summarized, with quotes 
exemplifying pattems/themes 
about service learning. Report 
to school district about value of 
service-learning for academic 
program — in paper presented 
to Board of Education. 



Brief summaries of what was 
observed; final report 
describing what students 
learned at their field 
placement and how learning 
was monitored by school 
personnel 

Report to school board 
science/math activities 
comparing math/science scores 
in the community of service 
learning students with those in 
non-service learning programs. 

Present to local teachers group 
to show them how math and 
science are applied through 
service-learning activities. 
Present to local school board to 
demonstrate student learning 
and to request more support for 
expanding service-learning 
opportunities for students. 
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ANALYSIS OF INFORMATION GATHERED 



Now that you have defined how to report the data, it is vital that you decide 
how the information will be analyzed. This is a difficult job, yet one of the 
most important. Explain briefly how you are going to analyze the 
information giving some detail as to which analysis techniques will be 



employed. 




Information Available 


Method of Analysis 


1. Interviews with students 
and community sponsors 
(I) 


I will summarize each 
interview and search for 
common or frequently occurring 
comments or themes. 


Z Observations from college 
students 
(I) 


I will review observation 
reports and describe the most 
commonly occurring themes or 
patterns. I will also report on 
the number of similar activities 
reported and how they relate 
to academic learning. 


3. Math tests covering 

service-learning experiences 
(II) 


I will compare the math scores 
from the service-learning group 
with the math class which did 
not have a service-learning 
dimension. 


4* Portfolios of student work 
in math and/or science 
(II) 


I will assemble the portfolios 
and describe information / 
applications of learning found 
in the typical case. I will also 
use the portfolios themselves to 
document student learning. 



III. DEVELOPING THE SELF-DIRECTED STUDY DESIGN 



Now the time has come. You should have enough information to put 
together your own action plan. Remember -- it is easy to do as long as you 
keep it simple, logical, and practical. Don't try to do too many things at once, 
nor make it too complicated. Revise your draft as much as possible until you 
feel comfortable with what you propose. The rest is easy! 
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Name: Rob Shumer 



Date: September 21, 1992 



SELF-DIRECTED EVALUATION PLAN 



Research Question 

How can I effectively monitor student learning activities in the community so I 
can be sure students are learning academic subjects? 

Information Needed 

- Information about my monitoring program 

- Information about what kind of activities students are doing in the community 

- Information about what academic subjects are being applied in various 

service-learning activities 

Information Users 

Myself : I can evaluate my monitoring plan to show my department head that I 
can justify awarding academic credit because I have good documentation of 
student work 

My department head : He can use my documentation to present to my principal 
to get additional funds to write up the monitoring process 

My principal : She can use the information to demonstrate student learning 
through service to possibly get funds from the school board to expand the 
program in our school 

Method for Collecting Information/Data 

I will use monitoring forms for 5 students in my program and write a short 
weekly review of how I keep track of their progress 

I will develop a form for the community sponsors to fill out each two weeks 
describing what students learn, especially academically related learning. 

I will also interview the students and the community sponsors. 

I will work with a local college to have evaluation students observe my 

students at their community service sites to document instances of academic 
learning 

Data Analysis 

I will write a report about how my monitoring system worked — whether it 
kept me better informed about what students were learning at their field 
sites 

I will write a report, using the community sponsor forms, describing the 
academic activities that took place. I will describe typical learning 
experiences for each student 

I will have the college students describe the subjects covered and typical 
situations where students applied academic learning at their community 
learning sites. 
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APPENDIX A 



USING PROGRAM GOALS TO PRIORITIZE QUESTIONS 

In this section you are to state what you hope your program accomplishes. 
The goal of your program may be to develop an attitude of service among 
your students or to show that service-learning contributes to the study of 
academic subjects. Whatever the case, try to describe the program goals of 
your service-learning program. 

Proeram Goals 



1. To teach civic and social responsibility to students through participation in 

service experiences in the community. 

2. To assist community members and agencies which are unable to receive help 

from government because of shortages of funding. 



ft 



3. To enhance the academic program by demonstrating the relationship 
between service-learning and traditional subject-matter study. 



4. To teach problem solving and group skills through service experiences. 
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CONNECTIONS BETWEEN QUESTIONS AND GOALS 



In this section you are asked to compare your questions with the program 
goals articulated above. Pick four questions which are most important to you, 
and then see if they correspond to any of your program goals. Please note that 
you do not have to select a question for your final version of the plan just 
because it matches with one of your program goals. You should pick 
questions simply because they are the most interesting and present the best 
chance of completion. 

Questions Goals 



a. How can I more effectively monitor student learning #3 

activities in the community so I can be sure students 
are learning academic subjects? 

1 . How can I measure what students learn through their # 1 ,# 3 , #4 

service-learning activities? 

c How can I show that students learn more math and #3 

science through participation in service-learning 
project? 



d. What real benefit is there for the community #2 

agencies which participate in our service-learning 
program? 



Hopefully, you see the relationship between program goals and specific 
questions developed. Now you must decide if you want to focus on program 
goals or else choose your own emphasis. In either case, now you must select 
the two most promising questions for the "study." 



APPENDIX B 



METHOD OF COLLECTING INFORMATION 



I. For this question I could interview students about what they learn each 
week. I could also ask them to write a journal about their service-learning 
experiences, especially asking them to cite examples where they feel they 
are learning academic subjects. I could also develop a form for community 
sponsors to describe the kinds of learning activities students do at their site. 
I could have a section dealing specifically with how students learn 
academic subjects. 



I could also involve students by having them interview each other. They 
could have assignments to observe other students at their community sites, 
recording what they do and making special notes to describe any 
applications of basic skills or academic subjects. 



EL I could do all of the same things as in Question I, plus I could give them 

special tests covering the topics they say they learn through their journals. 
I could ask my friend who also teaches math and science to give the tests to 
his students since they don’t do service-learning activities. 



I could also ask students to bring in examples of their work, especially those 
dealing with math and science problems. They could make a portfolio of 
their work, showing how they apply math and science subjects. Students 
could also prepare tests based on what they learn for other students in 
similar setting. 
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HOW I WILL ORGANIZE THE INFORMATION I COLLECT 



I. I will organize this information around the sources of the data. First, I will 
have personal responses which will include students and community 
sponsors. In this section I will have summaries of my notes. Then I will 
have written information from journals and evaluations from community 
sponsors. Last, if I have time or could get some college students to do 
observations at the community sites, I could have some documentation from 
someone else as to what kind of learning was taking place at the community 
learning site. 



II. This question could be organized in a similar fashion to Question I, with 
personal responses from students and community sponsors, as well as written 
information in journals. I would also have tests covering the science and 
math experiences. In addition, I would have the observational reports from 
the college students documenting what was being learned through the 
service-learning experience. 
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DEVELOPING A SUPPORT NETWORK FOR EVALUATION 



SELF-HELP FOR SELF-DIRECTED STUDIES 



One of the goals of the Serve-America research effort in Minnesota is to develop a 
local support network for practitioners who are doing self-directed studies of 
service-learning. Because many of those involved have limited experience in the 
formal research/evaluation process, it is necessary to establish a system to provide 
technical and personal assistance. Since there is little likelihood that large amounts 
of money will be flowing into the state for research, it is important to develop this 
network as inexpensively as possible. To make up for limited funding, we need to 
tap into our creative talents to develop and implement such a system which is 
supposed to become a model for the rest of the nation. 

Given this rather challenging task, our goal today is to begin the process of designing 
and implementing such a system. The first job at hand is to combine our collective 
brain power to produce ideas and strategies which can deliver the desired system. 

We know initially that representatives from the various Serve-America programs 
form the first ring of our support network. That means you! Please make sure that 
you have the names and phone numbers of at least five (5) people you can contact 
over the next nine months who are either going through the same process as 
yourself or know people who can help to guide you through the self-directed study. 
We have taken efforts to introduce you to others at today's meeting who have some 
common bond. Please take advantage of these introductions. 

Next, we hope to develop contacts at the levels of professional organizations, as well 
as our institutions of higher education, to expand and enhance the network. You 
will be given worksheets to help develop this second ring, so think about those you 
know who can support the effort. 

The State Department of Education, the National Youth Leadership Council 
(NYLC), the Higher Education Coordinating Board, and the Minnesota Office on 
Volunteer Services (MOVS) form yet another source of assistance in the research 
effort. Many experts are available who can help you to plan and execute your self- 
directed study. 

So let's spend the next 30 to 40 minutes thinking, brainstorming, and planning, so 
our network can begin today. May our efforts be rewarded with some of the best 
plans ever made on a Monday afternoon! 



O 
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RESEARCH NETWORK PLANNING WORKSHEET 



" There is nothing better, when you are feeling neglected 
To have lots of friends, who help you stay connected " 

Please use this worksheet to create a research assistance network for those who are 
engaged in the self-directed study program of the Serve-America project. 



Your 

Name: Phone: 

Address: 

Position: Title: • 

Program: Grade Level(s) 



What you can contribute to the research assistance network: 



Two people you know who are knowledgeable about research or evaluation: 



Name: Phone: 

Address: 

Position: Title: 

Program: ; Grade Level(s) 

Name: Phone: 

Address: 

Position: Title: 

Program: Grade Level(s) 





THE RESEARCH / EVALUATION NETWORK 



Please list the names of people who can serve as members of the 
research/ evaluation network. If you know what they know, write it down! 

PEOPLE IN YOUR REGION WHO KNOW SOMETHING ABOUT EVALUATION 



Name: 


Phone: 




Address: 

Position: 


Title: 




Program: 


Grade 


Level(s) 


What They Know: 
Name: 


Phone: 




Address: 

Position: 


Title: 




Program: 


Grade 


Level(s) 


What They Know: 

PEOPLE IN HIGHER 


EDUCATION WHO KNOW SOMETHING ABOUT 


EVALUATION 






Name : 


Phone: 




Address: 


Position: 


Title: 




Program: 


Grade 


Level(s) 


What They Know: 






Name: 


Phone: 




Address: 


Position: 


Title: 




Program: 


Grade 


Level(s) 



What They Know: 
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PEOPLE IN THE COMMUNITY WHO KNOW SOMETHING ABOUT 



EVALUATION 




Name: 


Phone: 


Address: 


Position: 


Title: 


Program: 


Grade Level(s) 


What They Know: 



Name: 


Phone: 


Address: 


Position: 


Title: 


Program: 


Grade Level(s) 


What They Know: 



PEOPLE IN A SCHOOL 


DISTRICT WHO KNOW SOMETHING ABOUT 


EVALUATION 




Name: 


Phone: 


Address: 


Position: 


Title: 


Program: 


Grade Level(s) 


What They Know: 



Name: 


Phone: 


Address: 


Posi tion : 


Title: 


Program: 


Grade Level(s) 





What They Know: 



ORGANIZATIONS WHICH HAVE PEOPLE WHO KNOW SOMETHING ABOUT 
EVALUATION 



Name: Phone: 



Address: 


Position: 


Title: 


Program: 


Grade Level(s) 


What They Know: 



Name: 


Phone: 


Address: 


Position: 


Title: 


Program: 


Grade Level(s) 


What They Know: 



OTHER PEOPLE WHO KNOW SOMETHING ABOUT EVALUATION 



Name: 


Phone: 


Address: 


Position: 


Title: 


Program: 


Grade Level(s) 



What They Know: 



This list is just the beginning. Please keep it in a safe place and remember to add 
names. You can always send new names to: 

Dr. Robert Shumer 
The Generator Center 

R-460, Vocational and Technical Education Building 
1954 Buford Avenue 
University of Minnesota 
St. Paul, MN 55108 



APPENDIX 



EXAMPLE OF RESULTS OBTAINED 
FROM A 

SELF-DIRECTED EVALUATION TRAINING WORKSHOP 



The following is a summary of results obtained from a recent Serve America training 
workshop on self-directed evaluation. It was conducted September 21, 1992 by the 
Minnesota Department of Education and the Generator Center of the Department of 
Vocational and Technical Education of the University of Minnesota. It is included to 
give you an idea of the types of evaluation questions asked by the participants, who the 
participants felt the top ten users of the evaluation information are, the types of 
information they felt should be collected (both quantitative and qualitative), and 
possible methods of collecting this data. It also includes several sample pages from the 
first draft of Minnesota's Resource Directory which was developed based on information 
gathered from the participants at this time. This information will be used by those 
working with Community Youth Service Programs. It includes a list of the evaluation 
workshop participants, recommended network evaluators, and college campus service 
and service-learning programs. 
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October, 1992 



(FIRST Draft: Please send us any additions and/or corrections.) 
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DRAFT #1 • ( 10 / 14 / 92 ) 

Serve America (September 21) Workshop Participants 



Last Name 




Last Name 




First 




First 




Phone 




Phone 




Address 




Address 




City 


South St. Paul 


City 


Hill City 


State 


MN 


State 


MN 


Zip 


55075 


Zip 


55748 


Title 


Director 


Title 




Program 


Roosevelt Breakfast Program 


Program 


Generator School 


Grade Lev 




Grade Lev 




Last Name 


- 


Last Name 




First 




First 


- 


Phone 




Phone 




Address 




Address 




City 


Mound 


City 


Minneapolis 


State 


MN 


State 


MN 


Zip 


55364 


Zip 


55414 


- Title 


Education Program Coordinator 


Title 


Consultant 


Program 


Westonka Community Education and 


Program 


Harrison Secondary and Minneapolis 


Grade Lev 


K-Adult 


Grade Lev 


7-12 


Last Name 




Last Name 




First 




First 




Phone 




Phone 




Address 




Address 


— 


City 


Hutchinson 


City 


Red Wing 


State 


MN 


State 


MN 


Zip 


55350 


Zip 


55066 


Title 


Youth Director 


Title 




Program 


At Risk Youth Community Service 


Program 


Red Wing Detached Youth 


Grade Lev 


6-12 


Grade Lev 
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DRAFT #1 - (10/14/92) 

Serve America Evaluation Network Recommendations 



Last Name 
First 
Phone 
Address 

City PineGty 
State MN 
Zip 55063 

Title Curriculum Director 
Agency/Prog Pine City District #578 
Grade Lev K-12 



Last Name 
First 
Phone 
Address 
City 

State MN 
Zip 

Title Principal Investigator 
Agency/Prog Drop Out Prevention Intervention 
Grade Lev - 



Last Name 
First 
Phone 
Address 

City Hinckley 
Slate MN 
Zip 55037 

Title High School Counselor 
Agency/Prog Hinckley District #573 
Grade Lev 10-12 



Last Name 
First 
Phone 
Address 

City Hutchinson 
State MN 
Zip 55350 

Title Business Manager 
Agency/Prog Hutchinson Public Schools 
Grade Lev K-12 



Last Name 
First 
Phone 
Address 
City 

State MN 
Zip 

Title Program Leader 
Agency/Prog U of M Extension 
Grade Lev statewide 



Last Name 
First 
Phone 
Address 

City Rush City 
State MN 
Zip 55069 

Title Education District Director " 
Agency/Prog Sl Croix River Education District 
Grade Lev 



O 
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COLLEGE CAMPUS SERVICE and 
SERVICE-LEARNING PROGRAMS 



If you w ould like to develop Youth Service collaborations between your school or organizations and colleges 
and universities, contact the following persons: 



State Contact 


Mark Langseth 1-800-366-6952 
National Youth Leadership Council 
1910 W. County Road B, Room 216 
Roseville, MN 55113 


Mesabi Community Cotlepe 


Joel Longie (218)723-4776 
Arrowhead Community College Region 
American Indian Student Tutoring Program 
Mesabi Community College 
Virginia, MN 55792 


Augsburg Cntl^o* 


Mary True, Coordinator (612)330-1775 
Service Learning Program 
Augsburg College 
721 - 21st Avenue S 
Minneapolis, MN 55454 


Bemidii Stato ^r n j Yfn jfy 


Angie Smith, Coordinator (218) 755-2988 or 4022 
Bemidji State University 
Bemidji, MN 56001 


Bethel CoMpp fi 


Steve Young, Coordinator (612) 638-6417 
Community Service Program 
3900 Bethel Drive 
St Paul, MN 55112 


liraincrd Community College 


Dennis Eastman (218) 828-2284 
Mentor Program 
College Drive & SE 4th 
Brainerd, MN 56401 


o 
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MINNESOTA DEPARTMENT OE 



EDUCATION 




Capitol Square 550 Cedar Street 
Saint Paul. Minnesota 55101 612/296-6104 

October 14, 1992 



Dear Evaluation Workshop Participants: 

Thank you for participating in the September 21st workshop on self-directed evaluation. We 
hope the day proved worthwhile and that the information received is beginning to help you 
develop an evaluation strategy for yourself. Enclosed is a compilation of the questions you 
asked and summaries of ideas you had on information needed, who the key users of information 
are and possible methods for gathering data. The ideas were excellent and we thought we should 
share them with you right away. 

As we continue the networking process, we would like more information from you. Would you 
please fill out the enclosed evaluation form and return it by October 30th. We would appreciate 
it if you would let us know when you have scheduled a date for your regional group to meet 
again. One of us will be glad to attend. If you have any questions about this, please call one 
of us at the numbers listed below. 

. We have also enclosed the first draft of the Network Directory. This includes names you 
suggested and the college campus contacts. The list of participants at the workshop are also 
included. 

We look forward to working with you this year. Please don’t hesitate to call if you need some 
help, or write about any problem or issue you encounter. Best wishes for a successful year. 
We look forward to seeing you soon at your local network meeting. 



Sincerely, 





Bob Crumpton 

Minnesota Department of Education 
612 296 2703 



Minnesota Department of Education 
612 296 1435 




Rob Shumer ” 

Generator Center 
University of Minnesota 

Calls at NYLC: 612 631 3672 or 1-800 366 6952 






EVALUATION FORM 



SELF-DIRECTED RESEARCH/EVALUATION PLANNING WORKSHOP 



The purpose of this evaluation is to determine the value of the workshop on 
planning a self-directed research project around the topic of service-learning. Our 
goal is not to immediately measure what you thought, but rather to have you tell us 
what has happened two or three weeks after the workshop was completed. That, in 
many ways, is a better indicator of what happened during the sessions. So please 
take this form and remember to complete it by Octobei30, 1992. Return it, by mail, 
to: 

Robert Shunter 

Generator Center, R-460 

Voc. and Tech. Education Bldg. 

1954 Buford Avenue 
University of Minnesota 
St. Paul, MN 55108 



L What steps have you taken to continue the planning process 
started on September 21st? Please attach a copy of your 
research/evaluation plan in its current state. 



2. What have you done to get any professional or technical 
assistance for your project? 



3. What areas do you feel you still need assistance to plan and 
implement your study? 



4. If you have continued to work on your research design, what 



RESEARCH QUESTIONS ASKED BY INDIVIDUALS 
Evaluation/Training Workshop 



o How will participants benefit and gain usable skills? 
o How will cooperating agencies directly benefit? 



o How does involvement in a community service class change students’ perspective of 
themselves, their relations at home, with peers, with other age groups? 
o How do students see themselves and the value of service? 



o How does our service project enhance the learning (educational' opportunties) for students? 
o How have students become valued citizens within the community? 



o-What tangible life long skills do youth gain? e.g., decision making skills, p.r., refusal skills, 
o Are youth gaining a sense of community involvement? 



o What are the changed perceptions of staff and community about youth? 
o What are the changed perceptions of the youth participants? 



o What are the strongest risk factors for kids as identified by kids? 
o What are the benefits from service-learning? 



o How can I monitor the student/mentor relationships? 
o What are the changes throughout the course of the project? 



o What life long skills will be learned by performing youth service project? 
o What did students gain from service? 



o What impact does youth service have on the community, the area learning center and the 
school district? 

o What currently is being done in the community by (school based programs)? 



o Do students feel that they are a valuable community member and that their service does have 
an important impact on those served? 

o Do students volunteer in the future, when they are out of school? (Follow up study) 





o What value do youth place on this experience? 

o What impact is made on elementary school children who participate in the program? 

o What impact does the combination of community work service and classroom involvement 
have 

on correctional clients? 

o What impact does community work service have on the community? 

o Can emotinally disturbed students be successfully involved in volunteer activities in the 
community? 

o What can youth teach senior citizens and senior citizens teach youth about writing and theatre? 



o How can we evaluate service work which is integrated into classroom assignments and 
activities? 



o What tangible life long skills are young people learning as a result of participation? 
o Are youth gaining a sense of community involvement and responsibility from the program? 

o Are we reaching our target group? 

__ o Are youth moving toward better citizenship? i.e., less destructive behavior? 



- 0 How 03X1 1 "[‘ore effectively monitor student learning so I can be sure students are learning 
academic subjects? 

o How does this program involve participants in development, decision making, 
implementation and evaluation? 

o From the student’s perspective, what was beneficial about being involved? 

o What percentage of students involved in the project continue the relationship with residents 
after the project is completed? 

o Did the youth become involved in other service activities? 

o How do I benefit from the program? 

o How do we promote consistent participation from volunteers? 

a How did teachers begin to integrate service-learning into the curriculum? 

o Will youth internalize a connection between in-service training and the actual service? 
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. INFORMATION USERS-TOP TEN* 



1. Program directors and staff 

2. Teachers 

3. Students/ youth/ youth advisory boards 

4. Administrators 

5. Government and community agencies 

6. Funders; e.g., National Commission-Youth Service, United Ways, foundations, city councils. 

7. Community education directors and staff 

8. Parents 

9. School Boards 

10. Area Learning Centers 

Others Mentioned: 

City Council, Mayor, Probation Officers, Group homes, National Corrections Assn., other 
schools, mentors, volunteers, Chamber of Commerce, specific community groups. 

* List is in order of frequency mentioned. 
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INFORMATION NEEDED 



QUANTITATIVE DATA: 

1. Numbers of youth involved 

2. Types of youth involved; e.g., those at risk, gender, ethnic background 

3. Numbers involved in these aspects: planning, implementation, evaluation. 

4. Time involved; hours of service 

5. Types of service 

6. Types of sites 

7. Numbers of teachers trained in service-learning 

8. Numbers of teachers integrating service-learning into curriculum 

9. Info on youth— attendance, scholastic records, problem areas 

10. Numbers of youth who earned credit for youth service 

11. Crime offence statistics 

12. Numbers of community members who participated 

13. Numbers of businesses that participated 

14. Numbers of youth who successfully completed projects 

QUALITATIVE DATA: 

1. What do participants want to leam? 

2. What measures insure involvement in decision making? 

3. Participants’ perspectives on program-strengths and weaknesses. 

4. Motivators and barriers to participation. 

5. How is service-learning integrated into the curriculum? 

6. What are the improvements at school? 

7. What are the changes in attitudes, self esteem of participants? 

8. What is the degree of satisfaction of participants and community agencies? 

9. What skills do youth want to leam? 

10. What did they leam? 

11. What were the changes in attitude of youth toward the community? 

12. What were the changes in attitude of the communtiy toward youth? 

13. What are the academic outcomes? 

14. What were the benefits for youth, teachers, community? 



. METHODS OF COLLECTING DATA 



1. Individual Interviews: 

o interviews of youth, students 
o interviews with recipients of service 
o interviews with participants and volunteers 
o interviews-quotations, statements from variety of people involved 
o interviews with parents 
o taped interviews 

o interviews-beginning and end of project 
o follow up interviews-3 years after project ended 

2. Group Discussions: 

o focus groups 

o focus groups (pre and post) 

o reflective part of youth advisory board and youth council meetings 
o dialogue nights 
o group meetings/discussions 

3. Records: 

o student journals 
o attendance records 
o student records— tests, grades 

o records of other groups; e.g., nursing home records, court records. 

4. Surveys/Questionnaires: 

o surveys of participants, other persons, agencies 
o surveys (pre and post) 

5. Observations/ documentation of experiences: 

o observations reported by teachers, program directors, staff, 
students, agency coordinators 
o video documentation by students 
o other means of documentation by students 






6. Media Reports: 



7. Other 



o news articles and photos 
o t.v. clips 

o school newsletters, papers 
o agency newsletters 



o final products 
o case studies 
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